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It has been suggested that comprehensive computer-based instruction might 
best be used to improve the basic skills of educationally disadvantaged students. The 
Computer Pilot Program of the Division of Computer Information Services of the New York 
City Board of Education was designed to investigate such claims. The Computer Pilot 
Program is an ongoing project designed to identify comprehensive CBI programs and 
implementation factors which can combine to positively effect the academic performance, 
attendance, and attitudes of educationally disadvantaged students. In the 1987-88 school 
year, the thirteen such programs were investigated by the Office of Research, Evaluation 
and Assessment of the Board of Education, including Autoskills, CCC, CCP, CNS, Degem, 
ESC, Ideal, PALS, PC Class, Plato, Prescription Learning, Wasatch, and Wicat. These 
were placed in twenty-six elementary and secondary schools throughout the New York City 
public school system. The investigation consisted of site observations, interviews with staff 
memebers and students involved with the programs, statistical comparisons of students' 
1987 and 1988 scores on citywide tests of reading and mathematics achievement, and 
questionnaires sent to the vendors of the CBI programs investigated. 

In general, the programs were v^W recieved by the staff and students involved 
with them, and thei use generally resulted in significant achievement gains. An inverse 
relationship was found between students' grade level and achievement gains resulting 
from program involvement. In other words, students in the lower grades tended to exhibit 
greater gains tha i students in the higher grades. This relationahip is summarized 
graphically in Picture 1, and should be kept in mind when mean scores for individual 
schools are examined. 

The characteristics of the programs investigated are summarized in Figure 2 
which shows the number of programs having particular characteristics (one square = one 
program). Almost all the programs involved both computer management and computer 
delivery of instruction, although one program was managed but not delivered via computer, 
and one was delivered but not managed by computer. All but two programs covered 
reading, language arts, and mathmatics, and many covered other subjects as well. Two 
programs involved just phonics instruction. Most of the programs evaluated used . 
task-analytic, learnlng-by-objectlves approach, but two Involved process-oriented, whollstic 
designs. The programs were about evenly split between being designed soley for 
remediation and full curriculum packages, and between being oesignea sDeciticaily for 
remedial populations and being targeted for general populations. Target populat.or.s 
ranged between kindergarten and adult. Most of the programs evaluated ran on computer 
networKS. These were divided between IBM, Apple II, and proprietary configurations. 

The report which follows summarizes these findings for each of the thirteen 
programs evaluated. 



FIGURE 1 

AVERAGE EFFECT SEES BY GRADE LEVELS 
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FIGURE 2 

CHARACTERISTICS OF PROGRAMS EVALUATED 
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USER SUPPORT 



AUTOSKIU-S 



courseware design: 
content 



approach: 



target population: 



management system: 
reporting system: 



fBedback to students: 
individualization: 

hardwaiB configuration: 

peripherals, ate: 
recommended use: 

support sendees: 

tBsted. 

mean difference scores: 
efliBct sizes: 

staff atHtudes: 
student attitudes: 

coordinator ratings- 
program content 
program design: 
program aesthetics: 

comments: 



remedial drill and practice, applications 
phonics 

(Icon also has mil range of reading, language arts, 

mathematics courseware) 

task-analytic and process oriented - 

based on subtypes of reading difficulty and automaticity 

theories 

educationally disadvantaged upper elementary and 
secondary 

ciagnostic/prescriptive, management by component subskills 
skill based, various reports available including latency 
and error rates, charts on screen and available to student 

immediate 

pacing, mode, timing of screen presentations 

64 MB Unysis Icon filesen/er networked to Unysis lcon2 

workstations 

printer 

4 thirty-minute sessions/week 

presen/ice and inservice training, maintenance contracts 

1 elementary, 1 junior high school, 1 high school 

significant at the p < .05 level 

1.3 

positive 
positive 



fair - excellent 
fair - excellent 
.'air - good 

theory-based phonics program, should not bo networked in 
non-local configurations, might be more effective when part 
of a complete language arts program 



Autoskills is a remedial phonics program for upper elementary and high school 
students that mns on a Unisys proprietary network. It was developed specifically for 
educationally disadvantaged students by a Canadian provincial educational authority. 
Based on tne assumption that comprehension automatically follows from fluency, the 
program does not specifically adress comprehension skills, but rather seeks to develop 
automaticity in d3C0dlng. The program is grounded In the theory that reading difficulty is 
comprised of subcomponent problems all of which a student may or may not have. 
Autoskills accordingly provides individualized training in three areas of readinp 
difficulties: visual matching, auditory-visual matching, and oral reading. Headphone's 
and taped speech are used in the audio sections of the program. 

An important part of the Autoskills program is its management component which 
monitors ftudents' progress and produces printed reports which detail their progress in 
terms of skill mastery and error and latency rates. This ir<formation can also be shown 
graphically and is available to students on request. Feedback concerning the correctness 
of individual answers is immediate. 

Autoskills was designed to be used for four 30-minute sessions per week. Its 
authors maintain that such scheduling will produce gains of one grade level for every 
twenty weeks spent using the program. 

The Usisys Icon network also comes with software packages offering language 
arts and mathematics instruction, word processing, graphics, educational games, and the 
Logo computer language. One school at which Autoskills was evaluated used Icon 
language arts, games, and word processing software in conjunction with Autoskills. The 
other two implementations did not. 

Autoskills was evaluated in an elementary school, a junior high school, and a 
high school during the 1987/88 school year. At the elementary school, Ajtoskills was 
used for five 45-minute sessions each week by fourth graders supervised entirely by 
paraprofessionals. At the junior high school, the program was incorporated into a larger 
resource room program for 54 seventh and eighth grade students and administered by a 
reading specialist. Students at this school spent three 20-minute sessions per week 
using Autoskills for phonics remediation. Their remaining resource room time was spent 
in silent reading, writing, ana using other Icon language arts software. An attempt was 
made to network the Autoskills system throughout an entire floor of the high schoo' so that 
computers could be placed in individual classrooms It was unsuccessful, and the school 
had to be dropped from the evaluation because of the resulting equipment problems. 
Indications are that Autoskills should not be networked in non-local configurations. 

Staff involved w'th Autoskills reported th?.t the program covered phonics very 
completely although those using the program by Itself thought that additional 
comprehension activities were needed. The staff interviewed also belioved the Autoskills 
training program was very useful. All agreed that the program's best feature was that it 
individualized instruction. Students interviewed concurred. "It gets us in the mood for 
working" one said. Results from the comparision of students' 1987 and 1988 reading 
scores are summarized below. These show that the Autoskills program was quite 
effective in improving students' reading scores at the schools where it was operational, 
achieving respectable average effect sizes of 1.3 and 1.4. The achievement gains at the 
Junior hign site were particularly impressive. Average acn'evement gams or 7.3 at that 
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school contrast favorably with the 4.7 average gain for all junior high students evaluated. 
Factors that may have contributed to this particular success include its integration into a 
complete language arts program the serious use mad of program feedback and the skill of 
the teacher in charge of the implementation. 
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courseware design: 
comsnc 



spproacti: 
target population: 

management system: 

reporting system: 

llBedback to students: 
individuaiization: 

hardware configuration: 



peripherals, etc: 
recommended use: 

support services: 



icSiBCI 

mean difliBrBnoe scores: 
effBct sizes 



staff attitudes: 
student attitudes: 

coordinator ratings- 
program oontBnt 
program design: 

program aesthetics: 

comments: 



full curriculum, tutorial and drill and practice 

mathematics, K - Algebra; 

reading, K- 12; 

language arts, K- 12; 

computer awareness, adult education 

task-analytic - competency based hierarchical sequencing 

general K - adult 

diagnostic/prescriptive, management by objectives; 
lesson/unit based 

objective based, various reports available 

immediate 
pacing 

Atarti ST w/ 1040 MB hard disk as sen/er networked 
to Atari microcomputers, or CCC MicroHost server and 
workstations 
printer 

3 - 4 twenty minute sessions/week 

on-site presen/ice and Inservice training, 
maintenance contracts, toll free hotline, free 
courseware updates, newsletter 

2 elementary schools 
significant at the p < .05 level 
reading - 1.6 
math - 1 .9 

positive 
positive 



good - excellent 
adequate - excellent 
fair- excellent 

excellent, well-developed program, currently being updated 
to include a full-color, more exploratory, environment 
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The Computer Curriculum Corporation's (CCC) basic skills packages provide 
individualized instruct-on in mathematics, reading, ana language arts for students In 
kindergarten through the tv/elth grade. Created by long-time computing in education 
specialist, Patrick Suppes, the program v^&s originally designed to operate from a 
Microhost mini-computer to networked v\«), KStations at local and remote sites. Problems 
with telephone lines and technological innovationshave led to a new version of the CCC 
program that runs on an Atari network. This new version includes ^;ll-color graphics, 
animation, and mouse-driven, experiential environments. The new version is being 
installed in one of the elementary schools evaluated, but was not in place at the time of 
the evaluation. 



The heart of the CCC program is a management component that monitors 
student performance, selects and presents individualized exercises, analyzes the 
responses to each exercise, and updates and stores each studenrs performance record. 
When a student achieves a high degree of success, the difficulty of the material is 
increased, ihe program provides the stu'jcnt with immediate feedback during each 
exercise and with a score for a series of exercises at the end of each lesson. Exercises 
are typically drill and practice consisting of multiple choice and short answer questions, 
although, as previously mentioned, the program is now being updated to include 
exploratory, enrichment exercises. 

The management component of the CCC program make.'j student progress 
reports available to teachers and/or administrators on request. Progress reports list each 
student's current position by grade level and lesson, and highlight low performance areas 
to facilitate grouping of students for further remediation. In addition, Ihey describe student 
progress over time and in each of several categories of mathematics, reading, and 
language arts skit's. 

The CCC program is organized into sections designed to be completed in 10 to 
20-minute sessions. Its authors suggest that students complete three or four sessions 
each week. 



During the 1987/86 school year, the CCC program was evaluated in two 
elementary schools. It had been in continuous operation at one oV these for eight years, 
where it shares space in the school library and is coordinated by the school librarian. In 
this school, 103 fourth graders used the program for basic skills remediation in both 
reading and mathematics. At the other elementary school, the CCC implementation did 
not become fully operational until February, 1988, due to equipment problems. In this 
school, 114 students in second, third, fourth, and fifth grade classes used the CCC 
program for four 20-minute sessions each week. The program is cordinated by a reading 
specialist and operated within a resource room. Classes meet for 40 minutes and 
students split their time between on- and off-computer activities. Classroom teachers 
accompany their students in the resource room. 
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What the teachers inten/iev^^d said they liked best about the CCC program was 
the amount of drill and practice it gave students, its immediate feedback and 
individualization of instruction, and that it gives educationally disadvantaged students a 
sense of independence and achievement. What they liked least about CCC was the 
limitations of the software. A specific weakness highlighted was its use of horizontal 
rather than vertlcie addition. The teachers Interviewed believed the CCC training tney 



received was quite useful, although they believed they needed more. Students 
interviewed stated that what they liked best about the CCC program was that it helped 
them to learn. 

Reading and mathematics achievement scores for both elementary schools 
where the CCC program was evaluated are given below. These show large gains in both 
areas (14.4 and 12.6 in reading; 44.0 and 41.1 in mathematics). That these gains were 
educationally meaningful is indicated by their large effect sizes (1.76 and 1.6 in reading; 
2.1 and 1.9 in mathematics). Results from these evaluations indicate that the length of 
time in operation is not a factor effecting the effectiveness of the CCC program. That both 
implementations were located within larger educational settings, hov/ever, may have 
significantly influenced the success of these implementations. 
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* - significant difference at p < .05 level 
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oou.3Pware design: 



apprc^ch: 
target population: 



remedial paper and pencil tutorial and drill and practice, 
computer-scored testing, enrichment using a variety of media 
ir'iCluding computer software 
basic, intermediate, and advanced matnematics; 
basic, Intermediate, and advanced reading; 
intermediJite and advanced language arts; 
functional competencies, science, social studies 
task-analytic ~ competency based hierarchical sequencing; 
"at-risk" high school and adult 



r/ianagefnent systefn: 



reporting systent: 



by sttjdent and teacher - computer-assisted, 
diagnostic/prescriptive, management by objectives; 
skill based; 

computer-scored mastery tests, reporting by student and 
teacher, quarterly reports to vendor required 



liaedback to students: 
individuaiizalion: 



computer scoring 

pacing; by teacher to an array of print, AV, and computer 
based materials 



haidware conflguration: 

peripherals, etc: 
rBoommerKled use: 



Apple He ^/ 20MB hard drive for scoring; 
other computers for running software 
scanner, modem, printer, AV equipment 
1 1/2-3 hours/day. 3 - £ days/week 



support services: 



presen/ice and in?ervice training, technical ajsistance 
available 



iBSiBa 

niean difference scores: 
effect sizes: 



2 high schools 
not significant 
reading - 0.4 



stafT attitudes: positive 
student attitudes: positive 



coordinator ratings- 
program content 
program design: 

prografn aesthetics: 



good - excellent 
excellent 

adequate - excellent 



comments: extremely time consuming for both students and staff, but 
extremely inexpensive in terms of inaterials and equipment: 
seems to be very dependeni on the qualiicy of staffing 



COMPARISON OF EFFECT SEES FOR CCC MPLB^ENTATIONS 
WTTH OVERALL AVERAGE EFFECT SIZES (ELBvlEKTARY) 
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The Comprehensive Competencies Program (CCP) uses a combination of 
printed materi?;;3, audio-visual aids, computer softwai'e, and a computer-based 
management system to provide instruction in both basic and functional skills to an older. 
at-risK student population. The basic skills component of the program provides 
remediation in reading, mathematics, language arts, social studies, and science; a 
functional skills component .eaches occupational and life skills. 

The CCP program is a learning-by-objectives program, it uses predesigned 
lessons, assignments, and tests that are individually sequenced by the teachers based on 
performance outcomes, allowing learners to proceed at their own pace. The program 
offers print-based instruction in basic, intermediate, and advanced (GED) reading and 
mathematics, and intermediate and advanced language arts, social studies, and science. 
It also includes enrichment and remediation activities involving an extensive variety c' 
print, audio-visual, and compulc-based materials. 

Students are placed in the CCP program and progress through it on the basis of 
paper-and-pencil mastery tests. Mastery tests are corrected by a card-reader and fed to 
the computer-based management component of the program which assesses students' 
progress and provides prescriptions for futher instruction. Record ke aping, however, is 
done by students and teachers on forms provided by the program. Teachers are also 
required to complete quarterly reports to the company so that they can keep track of 
Individual implementations. The program is organized around 45-minute sessions, but 
requirements for scheduling are flexible. Its authors recommend that students participate 
between 90 and 180 minutes per day, three to five days a week. 

During the 1987/88 school year, the CCP program was evaluated at two New 
York City high schools. At both schools, a comparable number of ninth grade students 
(50, 38) participated in the program for 80 minutes a day per subject, five days a week, 
receiving remedial instmction in both reading and mathematics. Because of the nature 
high school achievement testing, only reaoing scores were available for these students, 
'i hese were dramatically different between the two schools, most likely a result of quite 
dramatic differences in staffing at the two schools. At one school, where the student/staff 
ratio was 6/1 , students' achievement gains were well above high school averages. At the 
other, the student/staff ratio was 12/1 . and serious staffing problems developed. Students* 
reading scores actually declined slightly at this school. Reading scores for both schools 
are shown in the table at the end of this section. 

The teachers inten/lewed reported that the CCP program is strong in the areas 
of reading comprehension, computation, fractions, decimals, and percentages, and 
problem solving, and weak in the areas of vocabulary development, writing, language 
arts, problem comprehension, geometry, and algebra. What teachers said they liked best 
about the program was the flexible support structure It provides for their students. What 
they said they liked least about the program was certain of the printed materials and the 
complexity of its reporting system. All teachers interviewed, however, repoted that the 
training they received was extremely good and on-going. What the students interviewed 
said they liked best about CCP was that they were finally learning something. What they 
liked least about it was the length of time they were required to participate daily, which 
they believed was too long. 
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CNS 



cxxjrseware design: 

OOntBfTt 



approach: 
target population: 

management system: 

reporting system: 

feedback to students: 
individualizaUon: 

hardware configuration: 

peripherals, etc: 
recommended use: 
support services: 



mean difference scores: 
effect sizes: 



staff attitudes: 
student attitudes: 

coordinator ratings- 
program content 
program design: 

program uu'jtiiutic'j. 

comments: 



full curriculum, tutorial and drill and practice 
mathematics, 1 - Algebra; 
reading, 1-10; 
language arts, 1-10; 
can manage other software 

task-analytic - competency based hierarchical sequencing 
general 1-10 

diagnostic/prescriptive, management by objectives, 
lesson/unit based 

objective based, various reports available 

immediate 
pacing 

Apple II series computers networked thm Corvus, Lantech, 

Digicard, Appletaik servers 

printer 

optional 

preservice training, maintenance contracts, hotline 

1 junior high school 

significant for reading, but not for math 

reading - 0.8 

math - -0.2 

positive 
positive 



good 

fair 

good 

teachers and students interviewed were unhappy with the "low" 
quality of CNS software and their opinions were reflected in 
low performance gains, because the program can manage any 
Apple II software, an effort should be made to find more 
motivating and more effective software 
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The CNS program is a software package that combines a management 
component with full curriculum tutorial instruction and drill and practice in reading, 
language arts, and mathematics for grades one through ten. The management 
component of the program can also incorporate any software designed to run on Apple 
li's, allowing for a greater versatility in curriculum design. It is designed to run on Apple II 
microcomputers networked through Corvus, Lantech, Digicard, or Appletalk sen/ers. 

CNS is a learning-by-objectives program. The management component tracks 
students' progress by counting objectives completed, and produces printed progress 
reports that detail performance In terms of letsons and objectives mastered. In addition, 
students receive immediate feedback concerning the correctness of individual answers! 
No specific usage of the prograrr is recommended by its authors. 

During the 1987/88 school year, the CNS program was evaluated at one junior 
high school where it ran on a Corvus network located in the scjiool's computer lab. It was 
used for two 46-minute sessions each week by 67 seventh graders for mathematics 
remediation, and for three 45-minute sessions each week by 46 ninth-grade bilingual 
students for reading remediation. 

What both teachers and students interviewed liked best about the CNS program 
was its individualization of delivery, the positive reinforcement and opportunities for 
success it provides, and that it puts students in control of their own learning. What they 
liked least about it was the way it repeats incorrectly answered questtons, and the "low" 
quality of "much" of the soitware provided. The teachers inten/iewed believed that the 
CNS program was strong in basic reading and arithmetic skills drill and practice, but that 
there was little correlation between this and the school's curricula. In particular, the 
mathematics modules do not include a geometry unit, nor do the reading modules include 
interpretive literature, both of which make up important parts of students' regular 
classwork. Only the teacher acting as coordinator of the CNS program was given training 
in its use by the CNS staff. She found the training only moderately useful, and all the 
teachers interviewed believed that more training was needed. 

The 1987 and 1988 achievement scores for students participating in the CNS 
program are shown below. The results were better for reading, in which students 
evidenced significant gains (3.4 point gain, E.S. = 0.5). than they were for mathematics, in 
which students' performance actually declined slightly (-3.2 point decline, E.S. = -0.2). it 
IS interesting to note that while such low performance gains could validate the low 
opinions of CNS software held by participating teachers and students, it could al-^n be the 
case the software was ineffective because neither teachers nor students found it 
motivating. In any event, because the nature of the CNS management component allows 
it to incorporate any software that will run on Apple II computers, an effort should be made 
to find and test software more appealing to program participants. 
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DRP Scores: CNS 
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aCHOOL 
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N 


1987 


.1988 


MEAN OIF 


SD 


ES 


JHS 189 


7 


39 
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653.4 


-3.2 


20.8 


-0.2 



* - significant difference at p < .05 level 
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DEGEM 



courseware design: 
oontBnt 



approach: 
target population: 

management system: 

reporting system: 



feedback to students: 
individualization: 

hardware contlguratlon: 

peripherals, etc: 
rBoommended use: 

support sennces: 



remedial drill and practice 
mathematics, 1 - Algebra; 
reading, 1 - adult- 
language arts, 1 - adult; 
ESOL, typing, life skills, and more; 
authoring system 

task-analytic - competency based hierarchical sequencing 
educationally disadvantaged, elementary school thru adult 

diagnostic/prescriptive, management by objectives, 
lesson/unit based 

objective based, vanous reports available, including 
histograms 

immediate 
pacing 

DEC LS1 1 server networked to DEC workstations or 

MS-DOS microcomputers 

printer 

2 twenty minute sessions/week per subject 

preservice training, maintenance contracts, free courseware 
updates 



2 elementary schools, 2 junior high schools, 1 high school 
mean diflierence scores: significant at the p < .05 level 
eftect sizes: reading - 0.6 
math ~ 1 .0 



staff attitudes: 
student attitudes: 

coordinatDr radngs- 
program content 
program design: 

program aesthetics: 



mixed - positive 
mixed 



good - excellent 
good - excellent 
good - excellent 



comments: seems quite effective for the delivery of mathematics instruction at 
all levels, slightly less so for the delivery of reading instruction, 
students really do not like the lack of real keyboards and, as 
these are available in newer versions, It would make sense to 
use them 
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The Degem system, which is based on the Digital Equipment Corporation 
(DEC) PDP11/23 mini-computer, can support up to 64 student terminals, each equipped 
with a monochrome monitor and built-in touch keypad, it can be supplied with 
courseware packages providing drill and practice in basic mathematics, reading, and 
language arts, keyboarding. English as a Second Language, Algebra, and physics, as 
well as educational games. The approach taken assumes that the initial presentation of a 
topic, follow-up, and applications are best done by the teacher in the classroom, whereas 
the computer is an ideal medium for practice, evaluation, and testing. 

The Degem program includes a comprehensive management package with the 
ability to monitor and track individual performance. It automatically provides detailed 
diagnostics at every ability level to insure that students are working at their competency 
level. After diagnostics, a practice phase begins through which students advance at their 
own pace as they master the material. The Degem management component generates 
student and/c class performance records and comprehensive class and level reports 
which can be printed on request. When a student is having trouble with particular 
material, her teacher is alerted through special notations on these reports. 

Degem also offers an authoring system which allows teachers to create their 
own courseware, a test development facility for creating tests on-line, administrative 
routines, and programming tools. The recommended use of the drill and practice routines 
is for two 20-minute sessions per week per subject. 

During the 1987/88 school year, the Degem program was evaluated in two 
elementary schools, two intermediate schools, and a high school. In the elementary 
schools, the program was used by second through sixth graders for one 40-minute 
session per week. In one of these schools. Degem provided remedial instruction in both 
reading and mathematics; in the other, it was used by special education and remedial 
students soley for mathematics drill. One of the intermediate schools used the Degem 
program for one 40-minute session per week to provide basic reading and mathematics 
instnjction to general education students in the seventh through ninth grades. In the other 
intermediate school, remedial and special education students in the seventh and eighth 
grades received remedial mathematics and reading instruction in two 40-minute sessions 
per week. The high school evaluated likewise used the Degem program for two 
40-minute sessions per week to provide basic reading and mathematics instruction to 
Chapter 1 students in the ninth through tweith grades. Classroom teachers worked with 
their students in specialized computer labs in all the Degem implementations evaluated. 

What the teachers inten/iewed said they liked best about the Degem program 
was its diagnostic routines, its self-pacing features, that it provides students with 
immediate feedback concerning the correctness of their answers, that it reinforces skills 
learned in the classroom, that their students enjoy it. and that it is easy to use. What 
teachers said they liked least about Degem was the keypad, the lack of a science 
component, that it contained "too much repetition" and "not enough tutorial", ana that 
problems could not be kept on the screen long enough for them to answer students 
questions about them. Specific weakness highlighted were in the areas of phonics, 
spelling, geometry, logic, and problem solving, in addition, it was noted that the program 
forces students to add and subtract fractions horizontally, whereas this is done vertically in 
regular classwork. Many of the teachers particpating in the Degem programs were not 
trained In the use or ne system. Even those that were traineo oeiieve tnat more training 



would be very helpful. What the students interviewed said they liked best about Degem 
was the Immediate feedbacK, that it helped them concentrate, and that they controlled the 
system. They also thought that its graphics helped them to understand difficult concepts. 
What students liKed least about it was the keypad, its repetition of missed questions, its 
lack of explanations, its overall slowness, and that "it doesn't give you time to think". 

The 1987 and 1988 achievement scores of students participating in the Degem 
programs evaluated are shown in the tables below. In general, students evidenced 
significant performance gains in both reading and mathematics, although the program 
seems to have been more effective in the mathematics area than In reading. The average 
effect size in mathematics was 1.0. indicating that the program was educationally 
meaningful in this area; the average effect size of 0.6 for reading is less clearly 
meaningful. The patterns of results for the various grade levels mirrors results general 
found in the Computer Pilot Program taken as a whole. These describe an inverse 
relationship between grade level and achievement gains resulting from CBI use. It is 
useful to note, however, that use of the Degem program resulted in significant 
performance increases at all grade levels, indicating that, especially in the area of 
mathematics, it can be effectively implemented across grade levels. 



DRP Scores: DEGEM 



SCHOOL 

PS268 

JHS 210 
IS 252 

Jo^^n Jay HS 



GB 


N 


1967 


1968 


MEAN OIF 


SD 


ES 


3 


82 


50.3 


58.4 


8.r 


13.3 


0.6 


4 


78 


38.0 


44.9 


6.9* 


6.5 


1.1 


5 
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42.3 


64.4 


i2.r 


8.8 


1.4 


6 
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54.2 


57.2 


3.0" 


6.7 


0.5 


7 


43 


49.1 


54.3 


5.2" 


7.7 


0.7 


8 


48 


54.9 


58.2 


3.3* 


5.9 


0.6 


7 


64 
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0.6 
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C.4 
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1.8 
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10 
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59.9 


61.9 


2.0" 


7.0 


0.3 
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MATScOfes: DEGEM 



SQHQOL 
PS 225 

PS26e 

JHS 210 
IS 252 



GB 

3 
4 
5 
6 
3 
4 
5 
6 
7 
8 
7 
8 



N 

12 
71 
33 
23 
87 
77 
108 
113 
46 
43 
66 
21 



19EL 

508.0 
563.1 
688.0 
594.1 
572.0 
588.3 
612.8 
633.8 
617.9 
630.8 
617.8 
624.2 



1988 MEAN DIF SD 



541.6 
602.6 
603.7 
622.5 
593.1 
619.5 
647.7 
655.5 
628.5 
646.8 
621.9 
629.1 



33.6 
39.5* 
15.7" 
28.4* 
21.1* 
31.1* 
35.0* 
21.7* 
10.6* 
16.0* 
4.1" 
4.9 



23.1 
17.5 
17.6 
21.8 
33.8 
19.7 
23.0 
18.0 
16.8 
18.6 
15.9 
16.5 



ES 

1.5 
2.3 

0.9 
1.3 
0.6 
1.6 
1.5 
1.2 
0.6 
0.9 
0.3 
0.3 



* - significant difference at p < .05 level 
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ESC 



oourseware design: 
oontBnt 



approach: 
target population: 

management system: 



reporting system: 

feedback to students: 
lndlviduailzatk>n: 

hardware oonffguration: 
peripherals, elc: 
reoommended use: 

support services: 

tested: 

mean difterenoe scores: 
effect sizes: 



staff aUiludes: 
student attitudes: 

coordinator ratings- 
program content 
program design: 

pNogram aesthetics: 

comments: 



full curriculum, tutorial and drill and practice 
mathematics, K - 6; 
reading, 1 - 6; 
language arts, 1 - 8; 

task-analytic -- emphasis on higher order thinking skills 
general K - 8 

diagnostic/prescriptive, management by objectives, 
lesson/unit based; 

teacher can set mastery levels, interrupt sequence 
objective based, variou;> reports available 

immediate; on-line help 

pacing; branching for remediation; by teacher sequencing 

networked IBM PC, Apple II, or Tandy computers 
CD-ROM player, headphones, color monitors, printer 
3 thirty minute sessions/week 

preservice training, maintenance contracts 

1 elementary school 
significant at the p < .05 level 
reading - 1 .2 
math - 0.8 

mixed 
positive 



good 
good 
good 

interesting new program that bears watching as real world 
implementations help its authors revise and improve it; the 
relatively poor showing this program made in the 1987/88 
evaluation is almost certainly due, to a greater or lesser extent, 
to technical problems and scheduling problems at the 
implementation site; whether or not ESC will be more effective 
under more optimal conditions remains to be seen 
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The ESC program is a full curriculum program offering tutorial instruction and 
drill and practice in mathematics, reading, language arts, and social studies for students 
in kindergarten through the eighth grade. ESC is a newer learning-by-objectives program 
that emphasizes the development of higher order thinking skills. To this end, the program 
provides on-iine help and branches to special modules which model problem solving 
stiategies for students needing special help. Another innovative feature of ESC is that the 
entire program is contained on a CD-ROM disk from which Its full-color animation and 
sourKi sequences are accessed. 

The ESC program contains a management program which monitor? students' 
progress within the system and produces various reports detailing students' mastery of 
objectives, lessons, and/or units within the program. The placement and airection of 
students' progress, however, is orchestrated by the teacher(s) in charge of the program 
who are responsible for student assignment to the various program modules. 

ESC is designed to run on IBM PC, Apple II, and Tandy networks linked to a CD 
ROM player. Its authors reco...mend that it be used for three 30-minute sessions each 
week. 

The ESC system was evaluated in one elementary school during the 1987/88 
school year, where it was used by over 400 special education and Chapter 1 students in 
grades two through six for both reading and mathematics remediation. Three Chapter 1 
and two special education classes used the program, which is located in a computer lab 
and run by the computer coordinator, for one 45-minute session each week- eleven 
Chapter 1 and two special education classes used the program for two "5-minui6 
sessions each week. Because there were not enough computers for the students in the 
Chapter 1 classes, rotating schedules were been worked out in which some students 
each session were required to complete regular classroom assignments. Regular 
classroom teachers did not remain with their classes in the computer lab. 

ESC is a new system, and the implementation evaluated dunng the 1987/88 
school year suffered from frequent breakdowns involving bugs in both the haroware and 
software used by ESC. It is hoped that such problems will be elleviated as the program is 
increasingly tested and revised. 

What the teachers inten/iewed said they liked best about the ESC program was 
that it is individualized and self-pacing, that it fosters positive learning attitudes, that the 
lessons are colorful and Interesting and their students enjoy them, and that it pinpoints 
students' specific strengths and weakness and enables teachers to *ocus directly on 
these. What the teachers said they liked least about ESC was the frequent breakdowns 
they had experienced. When asked how they would improve the program and its use in 
their school, the teachers ovenA/helming replied, "Fix the system!" Other suggestions for 
program improvement included some means for automatically tracking and placing 
students, and more teacher training, support, and involvement. 

1987 and 1988 achievement scores for students participating in the ESC 
program are given in the table which follows. Reading gains averaged 9.8 points; 
mathematics gains averaged 19.5 points. V\/hile such gains are not at all outstanding 
among the elementary school implementations evaluated, both were significant at the p < 
.05 level, ana the average errect sizes or i.z in reaamg ana o.s m matnematics inaicate 
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that the ESC program produced effects that were educationally meaningful, it is likely, 
moreover, that equipment breakdowns and scheduling problems adversly effected the 
success of the ESC program during the evaluation year, and that scores might improve 
under better implementation corKlitions. 
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N 
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22 


632.8 


3666.7 


23.9* 


26.0 
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15 


542.0 


2559.7 


17.7* 


17.2 


1.0 




5 


18 


587.8 


4603.4 


15.7* 


23.6 


0.7 



~ significant difference at p < .05 level 
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IDEAL 



oourseM/are design: 
oontBfit 



approach: 
target population: 

management system: 



leporting system: 

fppdt>acktD students: 
individualization: 

hardware oonflguraUon: 

peripherals, elc: 
recommended use: 
support sen/Ices: 

tastod* 

mean diflererx» scores: 
efltet sizes: 

staff attitudes: 
student attitudes: 

coordinator ratings- 
program content 
program design: 
program aesthetics: 

comments: 



full curriculum, tutorial and drill and practice 
mathem:\tics, 1 - Calculus; 
reading, K - adult; 
language arts, K - secondary; 
writing, 3 - college; 

science, foreign language, programming, and more 
task-analytic - competency based hierarchical sequencing 
ger^eral K - adult 

diagnostic/presc.'iptlve, management by objectives, 

lesson/unit based; 

teacher can set mastery levels 

objective based, various reports available 

immediate 

pacing; by teacher sequencing 

Apple II series computers netv\/orked thru Con/us. Lantech, 

Digicard, Appletalk servers 

printer 

3 forty-minute sessions/week 

preservice training, maintenance contracts 

1 high school 

significant at the p < .05 level 
reading - 0.6 

mixed 
positive 



good 
good 
good 

program was more effective than most at the high school level, 
it runs on Apple II networks thus allowing schools with existing 
Apple labs to use them, it is also relativelv inexpensive in terms 
oi" both time and money; the program may oe culturally biased 
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The Ideal program is a comprehensive full curriculum courseware package 
designed to run on networked Apple II computers. It offers mathematics instruction for first 
grade through adult \&^^\s, reading for kindergarden through adult levels, language arts 
for kindergarden thro»jgh high school, and writing for third grade through college students, 
as well as lessons in Calculus, science, foreign languages, computer programming, and 
more. The program also contains an authoring system with which teachers can create 
their own courseware, and it will run courseware packages from Hartley, Random House, 
and Milikin as well. 

The authors of the Ideal program recommend that It be used three 46-minute 
sessions per week. Ideal is a learning-by-objectives program that monitors students' 
progress through tutorial and drill and practice exerdses, and allows students to work at 
their own pace. The program's management component tracks students' progress within 
the system by counting completed objectives, and chooses new lessons for them based 
on their thusly monitored performance. Teachers can change the prescribed sequencing 
of Ideal's lessons, however, as well as set mastery levels for each objective. The 
management component also enables teachers to call up and print out a variety of reports 
which detail students' mastery of individual objectives, lessons, and/or units. Whole class 
reports and class rosters are also available. 

During the 1987/88 school year, the Ideal program was evaluated in one high 
school where it ran on a Corvus network. The program was used by twenty eleventh and 
twelth grade Chapter 1 mathematics students for three 40-minute sessions each week, 
and by twenty-eight students in the ninth through twelth grades, five 40-minute sessions 
each week for basic skills remediation in reading, language arts, and mathematics. 

The teachers interviewed agreed that the Ideal program is especially StfOng in 
the areas of basic reading, writing, and arithmetic skills, and in the area of technical 
English, and they agreed that additional software in the areas of composition, social 
studies, and science would be very useful. One teacher also believed that certain of the 
modules were culturally biased. The teachers interviewed also agreed that they needed 
more training concerned with the use of the Ideal system, although they thought the 
training they had received was quite useful. V»/hat the students interviewed said they liked 
best about the Ideal program was that It allowed them to work at their own pace, and that it 
helped them to learn. What they said they liked (east about it was that it sometimes called 
correct answers incorrect. 

The following table shows ninth and tenth grade students' 1987 and 1988 
achievement scores in reading. Mathematics scores were not available at the high school 
level nor were reading scores above the tenth grade levei. Tne results snow siamficant 
pe.'formance gains for students in the tenth grade, and an average effect size of 0.6 for the 
program, which is quite high for the high 3choo! implementations evaluated and vvnich 
indicates that it had educationally meaningful effects. 




DRP Scores: IDEAL 



SCHOOL 


QB 


N 


1997 


1998 


MEANDfF 


SQ 


Tilden HS 


9 


5 


50.2 


53.0 


2.8 


10.6 




10 


17 


59.2 


63.2 


4.0* 


6.1 



0.3 
0.7 



- significant difference at p < .05 level 
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PALS 



oourseware design: 
contBnt 

approach: 
tarc«t populatioa- 



m&nagement system: 
reporting systsm: 

tieedback to stirdents: 
individualization: 

hardware configuration: 

psripherals, elc: 

reoommended use: 
support sennoes: 



tested: 

mean difference scores: 
effect sizes: 

staff attitudes: 
sbxJent attitudes: 

coordinator ratings- 
program content 
program design: 

program aesthetics: 

comments: 



remedial tutorial and practice 
adult remedial reading and writing; 
typing 

process oriented, multi-sensory, wholistic 

high school students and/or adults who read at or below 

a firth grade level 

by students and teacher 
student journals 

immediate 
pacing; by teacher 

4 IBM PS/2 Model 30 computers, 8 IBM PS/2 Model 25 
computers 

touch screens, speech adapters, headphones, videodisk 
players, typewriters, printer, consumables 

1 hour/day, 6 days/Week for 20 wepks 

preservice and inseivice training, maintenance contracts, 
site visits 

2 high schools 
not significant 
readirig - 0.3 

positive 
positive 



good 

good - excellent 
good - excellent 

this program was specifically designed *or educationally 
disadvantaged and at-risk high school students by John 
Henry Martin and is accordingly sort of an adult version of 
"Writing to Read", it is extremely expensive in terms of both 
equipment and staff time, and has not as yet produced 
results that would justify these, the development of follow- 
through programs, however, is worth exploring 
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The IBM Principle of the Alphabet Literary System (PALS) is a high technology, 
multi-media program that makes use of the IBM IntoWindow system, touch-screens, and 
CAV videodisk to provide individualized instruction in phonics, reading, and writing to 
high school students reading at a fifth grade level or less. 

Designed by John Henry Martin, author of IBM's highly acclaimed Writing to 
Read program for beginning readers, PALS is grounded in the notion that educationally 
disadvantaged students in particular can benefit from a multisensory, wholistic approach 
to learning. The PALS program accordingly uses videodisk to present a comic book 
narrative intended to dramatize the importance of the phonetic alphabet, audio headsets 
to accentuate the correspondence between written and spoken English, and 
touchscrf^ens to force concrete interactions between students and the program materials. 
The PALS program also includes writing and typing components operated on IBM PC 
computers distinct from the sophisticated machines ^hich run the phonics portion of the 
program. Students are require'i to work in pairs on its part of the PALS program, but 
individually on the writing and typing segments. 

Indeed, the PALS program is perhaps the most structured of the programs 
evaluated, in that it is designed to be administered in a highly exact manner. Specific 
equipment configurations are required in PALS labs, which accordingly can only be used 
for the PALS program. The labs are designed to be used by no more than sixteen 
students at one time, lab time Is strictly apportioned, and required (not suggested) usage 
is for 90 minutes a day, five days a week, for twenty weeks. The program is thus very 
expensive in terms of both equipment and time requirements. On the other hand, the 
PALS program contains no computer-managed comp^inents, rather leaves its 
management to the teachers and students involved. 

During the 1987/88 school year, the PALS program was evaluated in two New 
York City high schools. Both implementations included at-risk students in the ninth 
through twelth grades, and both adhered strictly to the PALS usage requirements. More 
than twice the number of students involved In the program at one school, however, 
participated in the program at the other school. 

What participating teachers stated they liked best about the PALS prograir. was 
that it motivated their students and that it included the wrling and typing corrponents. 
What teachers liked least about the program was its lack of "foliow-through". Twenty 
weeks, they believed, was a not long enough period to bring about the systemat.c 
changes in their students' behaviors and attitudes they needed for continuec' academic 
and vocational success. Accordingly, both schools are implementing "PALS I!' programs 
to provide such follow-through. What the students intcr'iewed said they li'^ed oesX about 
PALS was being able to use the computers and that they were learning to t\/pe, which 
they perceived to be a useful skill. What they likJ^J least about the program was the 
amount of time they had to participate each day an J its extensive writing requirements. 

Particpating ninth and tenth grade students' 1987 and 1988 achieviment 
scores in reading are summarized below. Mathematics scores are not available; at the 
high school level, nor are reading scores for the eleventh and twelth grades. In general, 
the results are disappointing. They reveal significant achievement gains only among 
ninth graders at one of the schools tested. Although the effect size of 0.9 found among 
these itudents Is very nigh Tor high school implementations ana eaucationaiiy meaningrj! 
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in itself, the effect sizes found among the majority of participating students were low e/en 
by high school standards. It would clearly be useful to discover what factors were peculiar 
to these students that might have influenced the significant improvements in reading they 
demonstrated. In particular, one would like to know whether such factors could be related 
to the PALS program. It is entirely possible, of course, that these students' achievement 
gains derived from sources other than PALS. 



DRP Scores: PALS 



SCHOOL 


GR 


N 


1967 


1989 


MEAN DIF 


SD 


ES 


M L King HS 


9 


17 


51.2 


52.2 


1.0 


5.5 


0.2 




10 


6 


51.7 


53.5 


1.8 


7.1 


0.3 


T Jefferson 


9 


10 


49.8 


55.1 


5.3* 


5.7 


0.9 




10 


10 


58.5 


49.4 


0.9 


6.4 


0.1 



- significant difference at p < .05 level 
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PC CLASS 



courseware design: 
content 



approach: 
target population: 

management system: 

reporting system: 

feedback to students: 
individualization: 

hardware configuration: 

peripherals, etc: 
recommended use: 

support sennces: 

tested: 

mean difliBrence scores: 
effect sizes: 



sijff attitudes: 
student attitudes: 

coordinator ratings- 
program content 
program design: 

program aesthetics: 

comments: 



any 

any courseware that mns on IBM PC computers can be 
adapted o the system by writing objectives and mastery 
tests for it, 
authoring system 

task-analytic - competency based hierarchical sequencing 
general, elementary school thru adult 

diagnostic/prescriptive, management by objectives, 
lesson/unit based 

objective based, various reports available 

immediate 
pacing 

IBM PC or cornpatible computers networked to IBM AT 

w/ 285 MB hard drive as sen/er 

printer 

none 

presen/ice training, maintenance contracts' 

1 elementary school, 1 junior high school 
significant at the p < .05 level 
reading - 1 .0 
math- 1.7 

positive 
positive 



very good - excellent 
very good - excellent 
very good - excellent 

PC Class IS essentially an open-ended delivery system, xr\us 
highly dependent on the software choosen by teachers; ;t makes 
sense, therefore, give teachers involved with it good training ;n 
not only the use of the system, but in software selection, and to 
allow them enough time to preview and select quality software 
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The PC Class program is an instructional management system that correlates 
learning objectives with coursework presented on IBM PC networks. It is an open system 
that allows for the learning-by-objectives management of any educational software that 
runs on the IBM PC microcomputer series. The software is loaded into the network file 
server and linked with specified objectives. Mastery of the package is then understood by 
t^.e systc.Ti tc .r.ear. mastery of the stated learning objectives. 

The PC Class system has two components. The diagnostic comDonent of the 
system allows teachers to construct their own tests or use ready-made ones to determine 
a student's mastery of any given set of instructional objectives. Test items can be selected 
from a system database. Learning objectives which correlate with New York City curricula 
are being developed. The prescription component of the system provides a menu o^ 
software that addresses the objectives a student has yet to master. 

Students are retested on a particular set of objectK/es until they are successfully 
mastered. An activity log records time on task, difficulty level, activities and objectives 
mastered, and test results. These can be pnnted as hardcopy reports. The reports do not. 
however, detail specific areas where student errors have been made. Because the PC 
Class system is open-ended, no particular recommendations are given for its use. 

During the 1987/88 school year, the PC Class system was evaluated in two 
schools, one an elementary school and the other a junior high school. In the elementary 
school, it was used by one bilingual and two Chapter 1 sixth grade classes for 
mathematics remediation, and by one bilingual and two Chapter 1 third grade classes for 
remedial reading. In the junior high school, the PC Class system provided instruction in 
reading and mathematics to 136 eighth graders. Students at both schools worked in 
pairs on computers. 

What teachers interviewed said they liked best about the PC Class system was 
its graphical representation of abstract concepts, its word processing software, that it was 
flexible and easy to use, and that it gave students a sense of excitement about learning. 
What they liked least about it was the lack of extensive dhll and practice software and the 
paired use of computers. Teachers believed that further training and more time to preview 
software would help them better utilize the program, and that a 1/1 student/computer ratic 
would help students better utilize it. Students interviewed agreed, citing shared use of 
computers as the thing they least liked about the program. What students said they iiked 
best about the program was its graphical display of abstract concepts, that it v.'as 
interesting and fun, that it helped them learn, and that they could work at their own pace. 

The achievement scores of students involved with the PC Class system are 
shown i- the tables which follow. Results show significant gains in both reading and 
mathematics. At the elementary school, the program proved substantially more effective 
for the delivery of mathematics instruction than it was for the delivery of instruction m 
reading, while at the junior high implementation, its use resulted in greater gams in 
reading than in mathematics performance. Because of the nature of the PC Class systerr.. 
such differences are most likely a result of individual software choices and not syster, 
related. In any case, effect sizes at both schools indicate that system use resulted in 
educationally meaningful improvements in students' performance in both mathematics 
and reading. 
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DRP Scores: PC CLASS 



SCHOOL 


GB 


N 


1987 


1968 


MEAN DIP 


SD 


ES 


PS 142 


3 


20 


29.4 


45.8 


16.4* 


17.4 


0.9 




6 


24 


46.0 


51 7 


6 7* 


F, 8 




JHS 141 


7 


10 


52^4 


57.2 


4.8* 


7.7 


n.6 




8 


106 


57.2 


63,1 


4.9 


6.7 


0.9 








MAT Scores: 


PC CLASS 








SCHOOL 


OR 


N 


1987 


1968 


MEAN DIP 


SD 


ES 


PS 142 


3 


25 


512.4 


568.9 


56.4" 


42.7 


1.3 




6 


28 


600.9 


636.7 


35.8* 


12.4 


2.9 


JHS 141 


7 


8 


620.9 


629.9 


9.0 


19.3 


0.5 




8 


91 


640.1 


649.0 


8.9 


14.4 


0.6 



- significant difference at p < .05 level 



COMPARISON OF EFFECT SEES FOR PC CLASS MPLaiEKfTATlONS 
WITH AVERAGE EFFECT SIZES 






0*1 



d4 




READING 



MATHaiATIC* 



AVERAGE 
ELBIEKTART 



AVERi^QE 
JR. HIGH 



PC CL4SS 



2^ 
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PLATO 



courseware design: 
content 



approach: 
target population: 



management system: 



reporting system: 

1iBedt>ack to students: 
individualization: 

hardware oonflguration: 

peripherals, elc: 
rBoommerxjed use: 
support sen/ices: 



testedi 

mean difference scores: 
effect sizes: 

staff attitudes: 
student attitudes: 

coordinator ratings- 
program content 
program design: 
program aesthetics: 

comments: 



full curriculum, tutorial, drill and practice, and experientai 
mathematics. 1-12; 
reading, K- 12; 
writing, 9- 12; 

science, social studies, lifeskills, and more 
task-analytic - competency based hierarchical sequencing 
high school - individual programs for general education, 
remedial, advanced placement 

diagnostic/prescriptive, management by objectives, 
lesson/unit based; 

teacher can set mastery levels, interrupt sequence 
objective based, various reports available 

immediate 

self-pacing; by teacher sequencing 

IBM PC or Apple II computers networked through LPDS 
server 

optional touch screens, printer 

2 - 3 forty minute sessions/week 

presen/ice and inservice training, maintenance contracts, 

hotline 

2 high schools 

significant at the p < .05 level 
reading - 0.4 

positive 
positive 



good - very good 
good - very good 
good - excellent 

demonstrated reasonably effecuve with high schooi stuoents, 
explicitly designed for use with such population, versatile 
and very comprehensive at that level, flexible hardware 
requirements and software capabilities, make this a good choice 
for high school implementations 
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The Plato program is a full curriculum program for older students that integrates 
tutorial, drill and practice, and experiential environments to cover almost a full range of 
typical high school course offerings. Plato courseware is available in mathematics at 
performance levels ranging from first through twelth grade, for reading at the kindergarten 
through twelth grade levels, and for high school writing, science, social studies, business, 
iifeskills. computer programming, and foreign language instruction, in general education, 
remedial, and advanced placement modes. 

The Plato system is run by a management component that places students 
within the program and traces their progress within it. This component will also generate 
printed reports detailing time on task, difficulty level, class standing, and the number of 
tries needed to achieve mastery for each activity the student has completed. The 
management component also offers administrative software for scheduling and 
budgeting, a spreadsheet, and a word processor, all of which are driven by the central 
server. 



Plato courseware uses graphics, sound, animation, and In certain modules 
audio-taped voice components delivered through headsets. It will run on either IBM PC or 
Apple II microcomputers networked through a LPDS file-server, allowing for its 
implementation in pre-existing computer labs. A Plato network can accomodate up tc 
thirty students working simultaneously at different levels, and remote sites can be serviced 
from the central computer via modem. When the microcomputers so networked include 
floppy disk drives, they can also be used to run standard IBM or Apple software. Plato 
software can also be used in stand-alone configurations via floppy disk drives. 
Suggested use of the program is for three 40-niinute sessions per week per subject area. 

During the 1987/88 school year, Plato programs were evaluated in two high 
schools. At one school, Plato was used to provide remedial instruction in mathematics 
and reading to ninth and tenth grade Chapter 1 students. Fifty-three students were 
enrolled in the mathematics program and sixty students participated in the reading 
program. Students in both programs used the Plato system for five 40-minuTe sessions 
each week. At the other school, Plato was used by more than 250 Chapter 1 and specia; 
education students in the .linth through twelth grades for basic skills remediation in both 
reading and mathematics. Students at this school used the system for two or three 
40-minute sessions each week. 

What teachers interviewed said they liked best about the Plato program was 
that it provided a variation from regular classroom routines but covered the required 
curricu'i- areas, that it was self-pacing and individualized, thus allowing students to 
repeat what they didn't understand, and that their students seemed more attentive and 
seemed to retain more information when using the Plato program. What teachers said 
they liked least about the Plato program was its lack of a good graphing module, that it 
returned students to the beginning of lessons they had not completed in a previous 
session instead of to where they left off. that it didn't give self-correction advise, and that 
other DOS programs could not be included on the network. They also reported that Plato 
was "teacher-unfriendly," that the manuals ana on-line instnjctions provided were unclear. 
When asked what they liked best about the Plato program, the students interviewed 
replied, "you don't have to write", "it's more relaxing, more fun. and not boring", and "you 
learn more Taster. They believed their involvement wixn Piato was neiping tnem to lea'n 
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and helping them with their other classwork. What the students interviewed said they 
liked least about Plato was the drill and practice exercises, although many said there was 
nothing they didn't like about it. One student complained that he couldn't eat in the 
computer lab. 

The 1987 and 1988 reading achievement scores for ninth and tenth graders 
participating in the Plato programs evaluated are given in the table which follows. 
Significant increases were fcjnd among participating ninth but not tenth graders. The 
small effect sizes for such increases, however, indicate that they were not particularly 
meaningful. None the less, the Plato impelementations v^re among the more effective 
high school programs. Because it has thus been shown to be reasonably effective, 
because it is very comprehensive and flexible, and because it was specifically designed 
for high school use. Plato would seem a good choice for high school implementations. 



SCHOOL GR 

Prospect H 9 
10 

S outh B ronx 9 





DRP Scores; 


PLATO 


N 


1967 


1988 


5 


57.2 


61.2 


17 


60,0 


62.1 


61 


67.2 


69.1 



MEAN DIF 


SO 


ES 


4.r 


7.7 


0.5 


2.1 


5.2 


0.4 


1.9* 


6.3 


0.3 



* - significant difference at p < .05 level 



COMPARISON OF AVERAGE EFFECT SEES 
FOR AVERAGE HIGH SCHOOL MPLB^EMTATIONS 
WTTH EFFECT SEES FOR PLATO 



0.3 r^:—, 

AVG PLATO 
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PRESCRIPTION LEARNING 



courseware design: 
contact : 



approach: 



target population: 
management systan: 



reporting system: 

feedback to students: 
individualization : 

hardware configuration: 

peripherals, etc.: 
reconmended use: 
support services: 

ttiSted: 

mean difference scores: 
effect sizes: 



staff attitudes: 
student attitudes: 

coordinator ratings: 
program content: 
program design: 
program aesthetics: 

conments : 



remedial tutorial, applications, practice 
mathematics , K-8 
reading K-8 

enrichment adult programs 

task-analytic — competency based hierarcnical 
sequencing 

parent involvement in student learning 
educational disadvantaged K thru 8 

by parent and teacher — computer-assisted, 
diagnostic/prescriptive, management by 
objectives; skill based 
skill based various reports available 

computer scoring 
pacing ; prescriptions 

Apple lie 20MB hard drive for scoring 
home-based Apple lie computers hooked to TV's 
scanner, modern printer 
2 eight week periods /year 

preservice and inservice training site visits 

1 elementary school, 1 junior high school 
significant at the p .05 level 
reading — 2*4 
math -- 0.8 

positive 
positive 



good-excellent 

good 

good 

highly effective program which may be especially 
useful for special education populations, it is 
likely that the multi-media approach inherent in 
this program together with links between learnini 
with the program and regular classroom learning 
made explicit by participating teachers 
significantly contributes to its success. 



The Prescription Learning program was originally designed as a take- 
home program premised on the notion that, because students spend a good deal 
more time at home than in school, the use of that time and the involvement 
of parents could result in meaningful increases in their achievement levels. 
In the elementary school implementation evaluated. Prescription Learning was 
used in this manner. Students were assigned computers to bring home for 
eight week periods. In students' homes, the computers were hooked to family 
televisions and students were asked to spend at least two 45-minute sessions 
a week completing program activities. At this school, twenty-six Chapter 1 
and nineteen general education students in grades four through six 
participated in the program. In the junior high school implementation 
evaluated Prescription Learning computers were placed on carts and moved 
into special education classrooms where fifty-six students in the sixth 
through eighth grades used them three to four times per week. Prescription 
Learning has now been updated to a network configuration, one implementation 
of which is being evaluated for the 1988-90 school year. 

Although variously implemented, the Prescription Learning program 
remains essentially the same in design. Students take tests which are 
scored by programs 's management component. The management component 
generates a "prescription" for each student. The prescription refers them 
to software and text materials covering areas in which they need 
improvement. Vfhen these are completed, the student is retested, and a new 
prescription generated. 

What staff members interviewed like best about the Prescription 
Learning program was its family involvement, and that it is self -pacing and 
encouraging. What they stated they liked least about the program was the 
amount of paperwork it involved. Although the program coordinator at the 
elementary school site took over another person and was never trained, staff 
members at the intermediate school site found the training given by 
Prescription Learning "quite useful". What students interviewed said they 
liked best about the Prescription Learning program was the puzzles and games 
it provides them with, that it made school "exciting and fun", and that they 
could take the computers home. They reported that there was nothing they 
didn't like about it. 

Results from the comparison of students' 1987 and 1988 achievement 
scores are shown below. They indicate that the program was very effective, 
especially for the special education students tested at the intermediate 
school. It is not clear to what degree this reflects the inverse 
relationship between achievement level and program effectiveness but further 
investigation of the use of this program with special education students is 
clearly indicated. It is also possible that the placement of this program 
in locations other thcui computer labs and its extensive use of text 
materials may have helped teachers involved with the program to make the 
links be\.ween the program and classroom learning more explicit and so 
facilitate the transfer of program gains to standardized testing. 
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DRP Scores PRESCRIPTION LEARNING 



SCHOOL 




n 


1 QR"? 

J.70 / 


1 QQQ 


MEAN DIF 


SD 


ES 


PS 58 


4 


11 


34.8 


40.1 


5.3* 


4.9 


1.1 




5 


20 


43.8 


59.6 


16.3* 


6.5 


2.5 






a 

o 


4 7 . J. 


^ A 1 
3 4*1 




4 . 8 


1 . 2 


IS 390 


8 


19 


43.1 


57.3 


14.2* 


4.8 


3.8 










MAT Score: 


PRESCRIPTION 


LEARNING 




SCHOOL 




N 


1987 


1988 


MEAN DIF 


SD 


ES 


IS 390 


8 


17 


6021 


621.5 


194** 


22.9 


0.8 



COMPARISON OF EFFECT SIZES FOR PRESCRIPTION LEARNING IMPLEMENTATIONS WITi 

AVERAGE EFFECT SIZES 




WASATCH 



courseware design: 



content: 



aK>roach: 
target population: 

management system: 



reporting S3rstem: 



feedback to students: 
individualization : 

hardware configuration: 



peripherals, etc: 
recannended use: 
support services: 



enrichment thru practice, 
experiential 

environments , computer-based 
tools 

mathematics , K-Algebra 
communication arts, K - 12 
science 

process or-..ented, wholistic 
general K thru 12 

by sequenced activities 
student and teacher can 
interrupt 

activity based, activities 
completed reported 

iiiunediate 
pacing 

IBM AT w 285 MB hard drive as 
server networked to IBM PC or 
compatible computers 
color screen printer 
none 

preservice and inservice 
training site visits for first 
5 weeks, free courseware 
updates 



tested: 

roe<m difference scores: 
effect sizes: 



1 elementary, 1 internediate 
school 

significant at the p .05 level 
reading — 0.7 
math — 0.6 



staff attitudes: 
student attitudes: 

coordinator ratings: 
program content: 
program design: 
program aesthetics: 

connents : 



positive 
positive 



excellent 
excellent 
good-excellent 

this program was less effective than 
others in terms of achievement gains, it 
may be because it is computer oriented, 
teachers do not know how ro make 
effective use of it. Extensive teaching 
is suggested. 
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The Wasatch program is designed to provide enrichment through practice 
and involvement with computer-based experiential environments and tools to a 
general kindergarten through ninth grade student population. The approach 
is process --iented and . The Wasatch wong lab for example provides 

students with state-of -th^-art computer-based writing tolls in word 
processor, speller^ style analyses and varies pre-writing programs and sets 
of sequenced writing activities designed to focus on particular aspects of 
good writing. The science components of the program likewise feature 
simulated science experiments and on-line data-base and a variety of 
computer-based measuring and calculating tools with which to complete them. 

The Wasatch program offers activity-based programs in mathematics for 
kindergarten through Algebra and in communication arts for kindergarten 
through the twelfth grade. The science components are designed for grades 
six through ten and offer modules in life sciences, physics and earth 
science. All activities incorporate full color graphics and animation. The 
program also includes the Logo programming lange a discovery oriented 
children's programming environment. 

The entire Wasatch program is run by a management component that backs 
student progress through sequenced series of activities. These sequences 
Gin be changed by a teacher and or interrupted by the student. The 
management component can denervate printed reports on request that detail 
student progress in terms of activities completed and time on task. The 
Wasatch program runs on networked IBM PC or PC compable microcomputers. No 
specific usage of the system is suggested. 
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had been inadequately trained suggests that this may bo the case, and 
perhaps indicates that more extensive teacher training should be given for 
Wasatch imp'^.ementations • It could also be that process-oriented programs 
demand greater teacher involvement than objectives based on programs and 
that this factor should be considered in implementation decisions. 

DRP SCORES: WASATCH 



SCBOOL 




N 


1987 


1988 


yfSAN DIF 


SD 


ES 


PS 126 


3 


15 


35. 5 


42.6 


7.1* 


9.9 


0.2 




4 


33 


31.2 


34.3 


3 . 2* 


6.6 


0.5 




5 


67 


50.0 


58.6 


8.5* 


8.5 


1.0 




6 


34 


58.0 


62.9 


4.9* 


7.2 


0.7 


IS 117 


7 


52 


5C.9 


52.3 


1.4 


11.3 


0.1 




8 


93 


53.5 


58.9 


5.4* 


5.9 


0.9 




9 


6 


59.7 


61.5 


1.8 


6.1 


0.3 










MAT SCORES: 


WASATCH 






SCHOOL 




N 


1987 


1988 


MEAN DIF 


SD 


BS 


PS 126 


3 


17 


554.4 


560.0 


5.6 


21.0 


0.3 




4 


35 


564.1 


579.4 


15. J* 


22.1 


0.7 




5 


65 


617.2 


642.5 


25.2* 


21.9 


1.2 




6 


34 


642.7 


670.6 


28.0* 


19.7 


1.4 


IS 117 


7 


55 


626.5 


621.4 


-5.0 


122.4 


-0.2 




8 


88 


621.4 


633.6 


12.2 


14.5 


0.8 



♦significant difference at p .05 level 
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WICAT 



courseware design: 
content: 



approach: 
target population: 
manag^nent system: 

reporting system: 

feedback to students: 
individualization : 

hardware configuration: 
peripherals, etc: 
reconmended use: 

minute sessions/weekly 



support services: 



tested: 

mean difference scores: 
effect size: 



staff attitudes: 
student attitudes: 

coordinator ratings: 
program content: 
program design: 
program aesthetics: 



full curriculum, tutorial and drill 
ana practice 

mathematics, K - Algebra II 

reading K - 12 

writing K - 6 

language arts 3-10 

typing, science, GED and more 

authoring system 

task-analytic — competency based 

hierarchical sequencing 

general K - 12 

diagnostic prescriptive, management by 
objectives activity based teacher can set 
mastery interrupt 

objective based various reports available 

immediate on-line help 
pacing by teacher sequencing 

WICAT System 1250, 1255 or 300 
printer 

elementary 2-5 thirty 

secondary 2-3 forty minute 
sessions /weekly 

preservice and inservice training 

maintenance contracts 

hotline 



1 elementary school 
significant at the p 
reading 1-5 
math 2-4 

positive 
positive 



good - excellent 

good 

good 



05 level 



conments: 



well developed tested and refined program 
to be extremely effective especially in 
mathematics 
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A prophetary multi-user system that supports up to thirty-five student 
workstations, WICAT uses dedicated terminals but offers the option of 
connecting IBM PC or Apple II series microcomputers through special adapter 
kits. When standard computers are used as terminals, they will support 
standard software in addition to the software provided by WICAT. 

The WICAT program is an older well developed tested and refined system 
that offers over 2,000 hours of courseware spanning. Virtually every 
subject typically offered in kindergarten through the twelfth grade. 
Instruction includes full-color graphics, animation, and sound, and in some 
cases, a human voice component delivered through audio headphones. The 
program provides an authoring system through which teachers an develop their 
own courseware and tests, and administrative software for scheduling and 
budgeting, spreadsheets, and a word processor. 

The heart of the WICAT program is an activity-based management 
component that uses computer adaptive testing to help determine students 
placement within the system. As each student progresses through the 
program, the management component tracts and monitors her progress and 
issues regular reports that show difficult levels, time on c^sk, and the 
number of attempts needed to master each activity, as well as the number of 
activities attempted and/or mastered and the class standing for each student 
suggested us of the program is for two to five 30 minute sessions per week 
at the elementary level, and for two to three 40 minute sessions per week 
per subject at the secondary level. 

During the 1987/88 school year, the WICAT program was evaluated in one 
elementary school where it was located in the school library. It was used 
by 107 Chapter 1 students in grades two through five for remedial 
instruction in reading and mathematics. The WICAT program was used for 
three 45 minute sessions per week in reading and for two 45 minute sessions 
per week in mathematics. Classroom teachers stayed with their students and 
helped them with their work on the computers. 

What the teachers interviewed said they liked best about the WICAT 
program was that it was highly motivating, non- threatening and 
individualized that it reinforced lessons learned in the classroom, and that 
it freed them to work individually with students in areas where they were 
experiencing difficulty. What teachers said they liked least about it was 
its lack of math word problems, the limitations of the word processor the 
low quality of so;me of the re&Iing stories and that they could not choose 
which lesjirons their students worked on. Teachers felt the training they 
received in svstem use, however was very useful. What the students 
interviewed said they liked heL about the WICAT program was that it helped 
them to learn and that it was fun. What students said they liked least 
about the program was that sometimes they felt it too difficult or boring. 

Reading and mathematics achievements scores for students participation 
in WICAT program are given below. Results indicate that students made large 
and significant gains in reading and very large and significant gains in 
mathematics indications re that it is a particulary effective program 
especially in the mathematics areas implementation factors that may have 
effected the programs success includes well trained and committed teachers a 
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low student/ staff ratio and good information or program feedback. 



DRP SCORES: WICAT 



SCBOOh 




N 


1987 


1988 


MEAN DIF 


SD 


PS 31 


3 


20 


36.3 


52.0 


15.7* 


10.8 




4 


21 


29.8 


36.2 


8.4* 


6.9 




5 


22 


39.8 


50.7 


11.4* 


6.5 










NAT SCORES 


: WICAT 




SCBOOL 




N 


1987 


1988 


MEAN DIF 


SD 




3 


20 


524.9 


569.2 


44.8* 


22. A 




4 


19 


554.5 


590.7 


36.2* 


16.1 




5 


23 


574.6 


611.0 


36.3* 


12.3 



ES 

1.5 
1.2 
1.8 



ES 



1.9 
2.2 
3.0 



^significant difference at p .05 level 



COMPARISOM OF EFFECT SIZES FOR WICAT IMPLEMENTATICMTS 
WITH OVERALL AVERAGE EFFECT SIZES (ELEMENTARY) 
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